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Paper-and-pencil tests are "fixed-item" tests in which all students answer the same 
questions. Fixed-item tests waste students' time because they give students a large 
number of items that are either too easy or too difficult. As a result, the tests give little 
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information about the particular level of ability of each student. With recent 
advancements in measurement theory and the increased availability of microcomputers 
in schools, the practice of using these tests may change. Computerized tests may 
replace paper-and-pencil tests in some instances. 

With computerized tests, each student's ability level can be estimated DURING the 
testing process and items can be tailored to this estimate of ability. Consequently, 
students can take different versions of the same test. These tests are called 
computerized adaptive tests or CATs. This digest offers some insights into the 
advantages and disadvantages of CATs. 

WHAT ARE THE ADVANTAGES OF CATS? 

In general, computerized testing greatly increases the flexibility of test management. 
Tests are given "on demand" and scores are available immediately. 

Neither answer sheets nor trained test administrators are needed. Administration is 
consistent. 

Test administrator differences are eliminated as a factor in measurement error. 

Tests are individually paced so that a student does not have to wait for others to finish 
before going on to the next section. Self-paced administration also offers extra time for 
students who need it, potentially reducing one source of test anxiety. 

Test security is increased because hardcopy test booklets are never compromised. 

Computerized testing also offers a number of options for timing and formatting. Timing 
options range from self-paced administration to item-by-item timing. Also, different 
formats can be developed to take advantage of graphics and timing capabilities. For 
example, perceptual and psychomotor skills that are nearly impossible to assess with a 
paper-and-pencil test can be readily tested on a computer. 

In addition to having the advantages of computerized testing, CATs increase efficiency. 
Significantly less time is needed to administer CATs than a fixed-item test since fewer 
items are needed to achieve acceptable accuracy. CATs can reduce testing time by 
more than 50% while maintaining the same level of reliability. Shorter testing times also 
reduce fatigue, which can be a significant factor in students' test results. 

CATs can also provide accurate scores over a wide range of abilities while traditional 
tests are usually most accurate for average students; CATs can maintain a high level of 
accuracy for all students. By including more relatively easy and more relatively difficult 
items in the item pool, CATs can accommodate the abilities of both bright and slow 
students. 
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WHAT ARE THE LIMITATIONS OF CATS? 

CATs should not be used for some subjects and skills. Most CATs are based on an 
item-response theory model, which assumes that all the information needed in selecting 
items can be summarized in one to three parameters that describe the item's difficulty 
for students who have different abilities. Many tests cover a number of different skills or 
topics, however. Specifications for traditional tests seek to ensure an even range across 
skills or topics. Most common CAT strategies do not accommodate such additional 
considerations. 

Hardware limitations further restrict the types of items that can be administered by 
computer. Items involving detailed art work and graphs or extensive reading passages, 
for example, are hard to present using the types of computers found in most schools. 

Another limitation of CATs stems from the need for careful item calibration. Since each 
student takes a set of items, comparable scores depend heavily on precise estimates of 
item characteristics. Therefore, relatively large samples must be used. A minimal 
number in a sample is 1 ,000 students; 2,000 is more common. Such sample size 
requirements are prohibitive for most locally developed tests. 

Finally, for CATs to be manageable, a facility must have enough computers for a large 
number of students and the students must be at least partially computer-literate. While 
the number of computers in schools continues to grow, many schools simply do not 
have the resources to use CATs as a standard practice. 

WHO IS USING CATS NOW? 

CATs are new and the number of companies and organizations using them is small. 
However, several prominent organizations are already using CATs. 
For example, for the past decade, the U.S. military has pioneered basic and applied 
research in CATs. One step in this research program is the development of a 
computerized version of the Armed Services Vocational Aptitude Battery (ASVAB), 
headed by the Naval Personnel Research and Development Center in San Diego. 
Administered to roughly a half million applicants each year, the paper-and-pencil 
version of the ASVAB takes three hours to complete while the experimental CAT 
version takes about 90 minutes. With the computerized version, an examinee's 
qualifying scores can be immediately compared with requirements for all available 
positions. 

Another test developed by military research laboratories - the Computerized Adaptive 
Screening Test (CAST) - was implemented in 1984. CAST was the first nationwide use 
of CAT. This 15-minute screening test gives prospects a quick but accurate estimate of 
their chances of passing the full ASVAB and of qualifying for enlistment bonuses. 
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As another example, two public school systems are forerunners in using CATs in the 
educational arena. In Portland (OR) Public Schools, CATs have been well received by 
examinees, test administrators, and test users. Montgomery County (MD) Public 
Schools has asked for approval from the State Board of Education to make its 
mathematics and reading CATs available to students as an alternative to the 
state-sponsored high school graduation examinations. 

WHICH BUSINESSES ARE INVOLVED IN CATS? 

The following six organizations are now involved in computerized adaptive testing: 
Assessment Systems Corporation markets the MicroCAT system, which runs on 
IBM-PCs and compatibles. MicroCAT is a complete authoring and administration 
system and includes routines for item analysis and item-pool development. The 
Montgomery County Schools CAT program is based on MicroCAT. 

WICAT markets software to support CAT developers, a battery of 45 tests, and custom 
CAT computer systems. Schools use Wicat's battery of CATs to screen and identify 
gifted and talented students. 

The Psychological Corporation markets a CAT version of the popular Differential 
Aptitude Test (DAT) to junior and senior high schools. It has versions for the IBM-PC 
and Apple II computers. 

American College Testing Program (ACT) is working on several computerized adaptive 
tests. ACT is developing training CATs for the Marine Corps and for college placement 
mathematics. It is also researching the development of a multidimensional CAT. 

The Educational Testing Service is working with the College Entrance Examination 
Board to develop and refine a CAT to aid in college placement. An initial version of the 
system is being used by about 20 colleges across the country. 

The American Institutes for Research recently completed a major revision of the Army's 
Computerized Applicant Screening Test (CAST). The CAST item pool was expanded, 
fairness analyses were conducted, item selection procedures were modified to increase 
accuracy at key points, and the feedback provided to examinees and recruiters was 
significantly improved. 

ADDITIONAL READING 

Green, Bert F., et al. "Technical Guidelines for Assessing Computerized Adaptive Tests, 
Journal of Educational Measurement. 1984, 21, 4, pp. 347-360. 
Kreitzberg, Charles, et al. "Computerized Adaptive Testing: Principles and Directions," 
Computers and Education. 1978, 2, 4, pp. 319-329. 
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Wainer, Howard. "On Item Response Theory and Computerized Adaptive Tests: The 
Coming Technological Revolution in Testing," Journal of College Admissions. 1983, 28, 
4, pp. 9-16. 

Weiss, David J. "Adaptive Testing by Computer," Journal of Consulting and Clinical 
Psychology. 1985, 53, 6, pp. 774-789. 

You can order a comprehensive bibliography with abstracts on CATs for $10.00 from 
the ERIC Clearinghouse on Tests, Measurement, and Evaluation. Call (202) 342-5060. 



This publication was prepared with funding from the Office of Educational Research and 
Improvement (OERI), U.S. Department of Education, under contract R-88-062003. The 
opinions expressed in this report do not necessarily reflect the position or policy of OERI 
or the Department of Education. Permission is granted to copy and distribute this 
ERIC/TM Digest. 
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